On the emergence of structure in behaviours evolved through embodied imitation in a group of robots

This paper describes research in which we model social interactions between artificial agents using real robots. We show that variations that arise from embodiment allow certain behaviours, those that are more robust to the processes of embodied imitation, to emerge and evolve during multiple cycles of imitation. We test 3 memory strategies: no memory, limited memory and unlimited memory, and experimental results appear to show that with limited memory, those behaviours are more likely to become dominant within the robots’ collective memory. Introduction Social learning, which enables individuals to learn from each other, is a powerful mechanism in social animals, including humans. An important form of social learning is imitation, in which an individual observes and replicates another’s actions. Imitation has been widely studied both by biologists and psychologists; biological research on imitation mostly focusses on its adaptive value for the organism, whereas psychologists are largely interested in the function of imitation and the mechanisms in which it plays a part (Zentall, 2001). There is continuing debate on the definition of imitation and whether it is unique to humans but what is not in doubt is that imitation clearly serves an important role in the development of social cognition in humans. For example, Dautenhahn et al reported that human babies are born with the ability to imitate a wide range of behaviours, including mouth opening and tongue protrusion (Dautenhahn et al., 2003). Meltzoff and Moore (Meltzoff and Moore, 1992) stated that human infants use imitation to enrich their understanding of people and their activities. Through imitation, humans are able to become part of a very complex social environment: human society. Imitation has also been seen as an important facet of cultural transmission; Dawkins argued (Dawkins, 1976) that imitation is a prerequisite for the evolution of culture, as it allows transmission of behaviours, with variation, between individuals. The study of imitation in robotics has received crossdisciplinary attention in recent years. In the context of robotics research, Bakker and Kuniyoshi (Bakker and Kuniyoshi, 1996) defined imitation thus: “Imitation takes place when an agent learns a behaviour from observing the execution of that behaviour by a teacher”. This definition hints at how imitation is implemented and is used in most robotics research. Skill acquisition by human or robot demonstration has been widely investigated ((Scassellati, 1999); (Mataric, 2000)). This approach holds the promise that we may be able to overcome the necessity to program every behaviour a robot may need to perform, as the robot can learn new behaviours through observing demonstrations of those behaviours. However, as stated above, as well as supporting skill transmission between individuals, in human society, imitation has a social dimension, allowing individuals to become part of a social community. Alissandrakis et al. (Alissandrakis et al., 2004) stated that imitation may serve as a stepping stone towards the development of social cognition in artificial agents as it can form social integration with other artificial agents or with humans. Imitation research in robotics might also usefully address the question of how culture emerges and evolves as a novel property in groups of social animals. In (Winfield and Erbas, 2011) we introduce embodied imitation as a method for modelling the emergence of behavioural ‘traditions’ in social agents. There has been some work examining the social dimension of imitation in robotics. Steels and Kaplan (Steels and Kaplan, 2001) stated that social learning can play a crucial role in initiating a humanoid robot into a linguistic culture. He used methods such as initiating open-ended dialogues among humans and robotic agents, in which social learning could be embedded. Billard (Billard, 1999) claimed that imitation can be used to enhance autonomous robots’ learning of communication skills. The sharing of a similar perceptual context between the imitator and demonstrator can create the necessary social context in which language can develop. Billard devised some experiments in which robotic agents were able to learn a proto-language by using imitation to match their environmental perceptions with observed actions. In this paper, we aim to show that by sharing a similar perceptual context, agents involved in multiple cycles of imitation can – in a sense – agree on the structure of the information that can best be transferred by imitation (that is, what can be imitated). Multiple robots are programmed to observe and imitate each other’s movement patterns and the imitated behaviours undergo multiple cycles of copying, in which they mutate because of noise and uncertainties in the real robots’ sensors and actuators. We observe that some movement patterns, which can be imitated with high fidelity, emerge and evolve in the group of real robots. Alisandrakis et al. (Alissandrakis et al., 2004) developed the ALICE architecture (Action Learning via Imitation between Corresponding Embodiments) to address the problem of imitation between dissimilar embodiments. They examined the rules of synchronisation, looseness of perceptual matching and proprioceptive matching in a series of experiments in which robotic arms with variably-sized and numbered joints imitate each other. They showed that patterns can be transmitted between simulated robotic arms and variations occur during these replications because of heterogeneities between the arms. They argue that these variations provide the evolutionary substrate for culture, as new behavioural patterns may emerge and be transferred between agents. In this paper, we describe a series of experiments in which real robots observe and imitate each other’s movement patterns. We show that even in an homogeneous group of real robots, variations occur during the imitation process that allow certain behavioural patterns to emerge and evolve during multiple cycles of imitation. These evolved behaviours can be copied with higher fidelity, as they are more robust to uncertainties in the real robots’ sensors and actuators. Imitation in Robots As stated above, we have used real robots to model the social interactions between agents. The motivation for using real robots rather than simulated agents or biological social entities for modelling is: • Real robots, with their less than perfect perception and actuators, provide natural variations in the imitation process which allow new behaviours to emerge and evolve. Using simulated agents in a simulated environment, we would have to control the degree and types of heterogeneities and noise, but this may preclude any emergent processes that are a part of imitation; the level of emergence in a simulated environment would be limited to the level of variance that is artificially introduced. • Data about the imitative activity, including the internal data and calculations of the robots, can easily be extracted and examined. This would not be the case if biological social entities (for example, people or monkeys) were used. • The implementation of imitation on real hardware makes clear how theoretical assumptions and hypotheses regarding imitation can be operationalised. Figure 1: A Linux-extended e-puck robot. The robots are fitted with coloured skirts, to enable them to ‘see’ each other. The yellow hat on top of the robot provides a matrix of pins holding unique patterns of reflective markers that allow the tracking system to identify and track each robot.